The present invention relates to methods and apparatus for processing image signals to remove redundant information thereby making the signals more suitable for transfer through a limited-bandwidth medium. The present invention specifically relates to methods and apparatus for use in video compression systems.
Many signal processing techniques useful in video compression systems are known. For example, digital encoding is often employed in processing television signals which are to be transferred over transmission channels since digital data streams are more immune to noise degradation.
In order to digitally encode a television signal, a significant number of bits, 4 or more, may be required to provide for an acceptable range of gray scale for each of the hundreds of thousands of separate picture elements (pixels) which form an image. Consequently, data rates for unprocessed digitalized television signals typically require a bandwidth greater than 40 megabits per second. If the communications link is an earth satellite, an unprocessed video signal typically occupies nearly the entire bandwidth of the satellite, with very few channels, if any, left over for other uses. A T1 communication channel is typical and has only a 1.5 megabit per second bandwidth. A practical yet effective way to reduce the bandwidth of digitalized television signals is needed so that fewer channels are required for transmission over a communications path and so that the quality of transmitted signals is maintained even when reduced bandwidth transmission is employed.
U.S. Pat. No. 4,302,775, assigned to the same assignee as the present invention, describes an improved scene adaptive coding technique which eliminates redundant information and thereby reduces the bandwidth.
The patent describes a single-pass digital video compression system which implements a two-dimensional cosine transform with intraframe block-to-block comparisons of transform coefficients without need for preliminary statistical matching or preprocessing.
Each frame of the video image is divided into a predetermined matrix of spatial subframes or blocks. The system performs a spatial domain to transform domain transformation of the picture elements of each block to provide transform coefficients for each block. The system adaptively normalizes the transform coefficients so that the system generates data at a rate determined adaptively as a function of the fullness of a transmitter buffer. The transform coefficient data thus produced is encoded in accordance with amplitude Huffman codes and zero-coefficient run length Huffman codes which are stored asynchronously in the transmitter buffer. The encoded data is output from the buffer at a synchronous rate for transmission through a limited-bandwidth medium. The system determines the buffer fullness and adaptively controls the rate at which data is generated so that the buffer is never completely emptied and never completely filled.
In the system receiver, the transmitted data is stored in a receiver buffer at the synchronous data rate of the limited-bandwidth medium. The data is then output from the receiver buffer asynchronously and is decoded in accordance with an inverse of the encoding in the transmitter. The decoded data is inversely normalized and inversely transformed to provide a representation of the original video image.
The U.S. Pat. No. 4,302,775 reduces redundancy by employing intraframe coding techniques utilizing intraframe comparisons of cosine transform coefficients. While the patent provides significant improvement over other techniques, there is a need for even greater compression.
In addition to intraframe coding techniques, interframe coding techniques have been used to reduce the rate required for video transmission. Typically, each video frame is held in memory at both the transmitter and the receiver and only frame-to-frame changes are transmitted over the communication link. In contrast to intraframe coding schemes in which the quality of coded images is dependent upon the amount of detail in each single image frame, the quality of the coded image in interframe coding is dependent upon the differences from frame to frame. Frame-to-frame differences are often referred to as "motion".
Interframe coding techniques are broadly classified into two categories, namely, spatial domain coding and transform domain coding. In one real-time interframe spatial-domain coding system, spatial domain data is threshold processed to obtain and store frame difference signals in a transmitter buffer. The threshold value is adaptively determined as a function of the transmitter buffer fullness. That system has not been entirely satisfactory because a wide range of threshold values is employed and results in a displeasing breakdown of the reconstructed image. In order to eliminate the image breakdown, both spatial and temporal subsampling has been proposed. Although these proposals have resulted in some improvement, they have not provided a system with fully satisfactory compression.
Interframe coding using transform domain coding has not been widely employed. In most proposed systems, the high cost of implementing a real-time spatial-to-transform domain transformer has made transform domain coding seem impractical. Nonetheless, several interframe transform coding systems have been proposed.
A conditional replenishment transform video compressor is described by Harry W. Jones, Jr. in the article "A CONDITIONAL REPLENISHMENT HADAMARD VIDEO COMPRESSOR", SPIE Vol. 119, Application of Digital Image Processing, (IOCC 1977), pp. 91-98. In that system, the transform coefficients for one frame are stored and compared with the transform coefficients for a subsequent frame. If the frame-to-frame threshold coefficient difference exceeds a threshold, the picture image or subimage is considered to have changed. The coefficient differences are measured relative to preselected vectors of the array of transform coefficients.
Another interframe transform coding system is described by J. A. Roese, W. K. Pratt, and G. S. Robinson, in an article entitled "Interframe Cosine Transform Image Coding", IEEE Transactions on Communications, COM 25, Nov. 11, 1977, pp. 1329-1339. In that system, frame-to-frame differences are employed as part of a differential pulse code modulation predictive coding technique. The transform coefficients are predicted using a fixed, first-order linear prediction function.
Another transform domain coding system is described by David N. Hein and Harry W. Jones, Jr., in an article entitled "Conditional replenishment using motion prediction", SPIE Vol. 207, Applications of Digital Image Processing III, (1979), pp. 268-277. That system detects and sends only the changed portions of the image. The receiver uses the data from the previous frame for the non-changed portion. Such systems are known as conditional replenishment systems, since only the changed data is sent (replenished). In the Jones system, the frame-to-frame difference analysis to determine change is carried on in the spatial domain. The conditional replenishment technique of Jones does not achieve fully satisfactory data compression.
While many different signal processing systems have been proposed, including those described above, none of them have provided fully satisfactory and sufficient data compression.
Accordingly, there is a need for improved signal processing methods and apparatus for data compression systems.